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ABSTRACT 

The reliability of space related assets has been emphasized after the second loss of a Space Shuttle. The intricate nature 
of die hardware being inspected often requires a complete disassembly to perform a thorough inspection which can be 
difficult as well as costly. Furthermore, it is imperative that the hardware under inspection not be altered in any other 
manner than that which is intended. In these cases the use of machine vision can allow for inspection with greater 
frequency using less intrusive methods. Such systems can provide feedback to guide, not only manually controlled 
instrumentation, but autonomous robotic platforms as well. This paper serves to detail a method using machine vision 
to provide such sensing capabilities in a compact package. A single camera is used in conjunction with a projected 
reference grid to ascertain precise distance measurements. The design of the sensor focuses on the use of conventional 
components in an unconventional manner with the goal of providing a solution for systems that do not require or cannot 
accommodate more complex vision systems. 
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1. INTRODUCTION 

The reliability of space related assets has been emphasized after die second loss of a Space Shuttle. The intricate nature 
of die hardware being inspected often requires a complete disassembly to perform a thorough inspection Re- 
qualification of the hardware for space flight is sometimes prohibitively expensive and time consuming, which can be 
difficult as well as costiy. In these cases the use of machine vision can allow for inspection with greater frequency 
using less intrusive methods. A group of areas on launch vehicles that would benefit from such inspection was 
identified and common barriers noted. Several measurement techniques were evaluated and compared against the 
environmental and performance requirements defined. A measurement technique using simple triangulation methods 
was adapted to accommodate these requirements. A miniature robot was developed as a platform to test the vision 
system. A test environment, appropriately sized for the robot, was constructed and the vision system was used to 
autonomously navigate the environment The results were evaluated and further testing conducted to define the 
miniaturization potential of the system. 

2. ENVIRONMENT AND PERFORMANCE DEFINITION 

Current plans for next generation space craft provide insight to possible environments requiring autonomous or semi 
autonomous inspection. An ideal inspection area was considered in a design for the X34 program employing a ‘skin’ 
raised off the spacecraft with a 10-12 inch separation creating a cavity that could be used to flow coolant Potential 
inspection areas identified during interviews with representatives of the Space Shuttle Main Engine (SSME) office 
included areas around turbine fan blades in the High Pressure Turbo Pump (HPTP), the “injection forest” where 
propellant and oxidizer enter the combustion chamber, and plumbing such as a cooling manifold around the nozzle. 

The physical environment to be navigated was defined as follows: constant lighting, curved walls, confined spaces with 
two to twelve inch length corridors. The width of the corridor would not be considered and for demonstration purposes 
would be defined by the platform developed to carry the vision system. Since miniaturization remains a goal the 
prototype development would be required to show a path to miniaturization to be developed in a follow-on effort. 
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Performance requirements include avoidance of curved surfaces and comers while maintaining a measurement error 
less than one pixel within the specified range of two to twelve inches. An additional performance requirement for the 
demonstration was to operate without a tether. 


3. DESIGN SOLUTION 


3.1 Approach 

Investigation of other solutions to simplify navigation techniques produced useful results. Commercial range finders 
are widely available and relatively accurate. Unfortunately most are designed for outdoor use and even those 
specifically for indoor use employ time of flight (TOF) techniques not effective for accurately measuring distances less 
than two feet. 

Range finders using triangulation dominate simplified distance navigation devices. One notable effort [1] employs a 
laser line projected onto die target. This research used deformation in the laser line to identify and alert an automobile 
driver of his proximity to an object using die specific example of a curb. Alternative distance measurement techniques 
using a single camera also include pixel dithering techniques [2] and an optoelectronic technique [3] for target 
recognition. Of the techniques studied the range finder was determined to be the simplest and easiest to modify for a 
given set of environmental parameters. The basic concept of triangulation is illustrated in figure 1 [4], 
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Figure 1 . Illustration of the basic concept of triangulation. 


The distance from the camera, point C, to the target, point T, may be calculated by using the Law of Sines: 
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where A represents the distance between the laser, point L, and the camera, point C. 


3.2 Technique 

Simple triangulation cannot account for curved surfaces, angles, or comers. The method using a laser line [1] is 
impractical for this application due the impact of increased computational requirements on size. Also, in an 
implementation the distance B, angles a, and b are very difficult to precisely define. A technique for obtaining range 



was demonstrated using a modular robotic testbed [5] will be briefly discussed. The camera with the laser mounted 
directly above, were configured as shown in figure 2. The laser and the optical axis form a vertical plane with point P 
as the target. 



Focal Plane 

Figure 2. Application of triangulation with simplified calculation 


Using similar triangles and placing the origin of the coordinate system at the focal point we have 
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In order to find x, the projection of point P onto die optical axis, two calibration points, PI and P2, are used. Given that 
points xl, x2, ul, and u2 are known, (4) may be derived. 
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where N, d, and k are known after calibration. 



In this manner Nguyen was able to calculate the range, x, without knowing the focal point of the camera or the distance 
d, angle a*, or angle bj shown in figure 1. This calibration technique works well over a wide range with accuracy 
decreasing as distance increases [5], 

While this technique achieves basic navigation, the confined space outlined in the environmental requirements requires 
distance measurement error of less than one pixel. The miniature lens dictated by the environment allowed using the 
technique in [5] without modification. However, the well-defined environment for this effort allowed a modeling 
technique to be performed prior to navigation. A combination of the two techniques described above was used to obtain 
real time range and environment information at the desired accuracy without the need to know d or angles a* and bj. 

In order to accommodate the environment defined above, the concept of a 3 x 3 grid of spots projected by a small laser 
onto die target was developed. Incorporating a simple prism or diffractive grating onto the laser allowed the design to 
remain compact while accounting for curved surfaces. Figure 3 is an illustration of the application of triangulation 
using a simple diffractive prism projecting multiple calibration points onto the target. 



Figure 3. Application of triangulation with the addition of a simple diffractive grating . 


Using equation (1) the range for each point, pi , may be calculated using 
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Information from spots adjacent to the center point permits the calculation of a rough shape of the target surface. Only 
the center point range need be calculated. Curved surfaces and comers may be avoided by calculating distances relative 
to die center using spatial distortions of the grid. 


4. IMPLEMENTATION 


4.1 Initial Implementation 

Evaluation of the feasibility of the concept was performed using the National Instruments software Lab VIEW® in 
conjunction with the IMAQ® tool suite. This approach allowed rapid development of a concept into a working system 
and proved ideal for prototyping the vision system. 

The assumption that lighting conditions are constant throughout the environment is reasonable as the inspection process 
is being performed in a controlled setting. This condition allows for the simplification of the image processing required 
to extract usable data from the image. This assumption is extended and the resulting requirements of the system as 
implemented include that the intensity of the laser exceed the intensity of the target in the image. With these conditions 
satisfied an intensity threshold allows for the projected pattern to be easily discerned from objects in the environment 
Each point to be measured in the image falls within a known region when the target is in the specified range. 
Performing edge detection in the regions of interest reveals the location of the projected pattern and allows the target 
position to be known. Should the edge detection fail to indicate a location within the region of interest, the 
measurement result is returned as off-scale. 

The relationship between target distance and point of interest in the image is inherently nonlinear. This non-linearity is 
compounded by the spherical aberrations introduced by the lens of the camera. While it is possible to flatten the image, 
allowing coordinates in the frame to correspond to real world measurements, it is a costly endeavor in terms of 
processing time and power. A simplified approached would create a linear approximation for the sensor output 
allowing rudimentary calculations to correlate sensor output to approximate target distance. This approach, while 
giving a reasonable target distance, introduces further error into the system and is unacceptable for navigating a 
confined space. 

The approach taken in implementing this system was to account for the non-linear relationship with a reasonable model 
of the data that would provide an acceptable error across the range. A set of precision, linear stages was used to provide 
reference measurements and the coordinates of the point of interest in the image were correlated to the actual 
measurements. A curve was fit to the data and the mean-squared error was evaluated to determine if the model was of 
appropriate accuracy. The sensor output allowed for single pixel measurements and resulted in the requirement that the 
model never vary from the actual measurement by half of a pixel in either direction. Through experimentation it was 
found that a sixth order polynomial could be consistently fitted to the data with the desired accuracy. The error between 
the model and the calibration data is shown in Figure 4. The cyclical nature of the error was found to be consistent over 
many calibrations and can be attributed to the discrete nature of the imaging system. This figure depicts this type of 
error well. The angle b, as defined in Figure 1, of the laser in relation to the camera, was only slightly less than ninety 
degrees limiting the vertical movement of the reference in die image to 6 pixels. As the target distance approaches a 
value accounted for by one of die discrete points associated with a pixel value in the image, the error approaches zero. 
As it moves away from this zero point the error increases until the midpoint is reached between the two adjacent pixel 
values. At this point the error begins to decrease and approach zero once again. It is not possible to account for this 
error in die model as the relationship between the error and the target distance is not one to one. It is then necessary to 
limit the error introduced by this step so as not to further degrade the accuracy of the output. In practice, the reference 
in the image was made to traverse as much of the height of the image as possible. While this lessens the effect of the 
error on the outputted measurements, it is still present in the system. 


4.2 Addressing Portability 


Lab VIEW® provided a platform for prototyping and experimentation; however it did not lend itself to a portable 
implementation of the software. The code, originally implemented in the graphic-based language of Lab VIEW®, was 
written using the same algorithm and in the more portable form of the C language. C compilers are common for a 
number of platforms including Texas Instrument’s TMS3206416 digital signal processor. (DSP) This processor was 
used to demonstrate the code in its new form. 

The DSP used is found in an evaluation board purchased from Ateme, a Texas Instruments partner, and also provides a 
frame grabber. The output of the frame grabber is made available to the DSP in the UYVY 4:2:2 format The first step 
in processing a frame of data is to strip the image of the chrominance planes. A black and white camera was used. 



Figure 4. Vertical measurement error introduced by mathematical model 


In die instances that good data is found in a region of interest, the fitted polynomial is employed to calculate the target 
distance in that region. The Newton-Raphson iterative method was implemented to determine the root of the 
polynomial offset by the location of the point of interest within the frame. This method was ideally suited for this case 
inthat the p olynomial was known to behave well in the neighborhood of the sensorrange. F urthermore, N ewton- 
Raphson provided die desired result with a minimum of computation making it ideal for implementation on a variety of 
low-end processors. As this was only a demonstration of the technique the output of the system was overlaid on the 
image and displayed. All points obtained during measurement were also indicated on the image to provide feedback 
during development Ultimately, the output of the system will take a form suitable to the target application. 

43 Extension of Algorithm to Three-Dimensional Orientation 

Also addressed in this implementation was the orientation of the target in three-dimensional space. The projected 
pattern was extended to allow for multiple points to be measured in the vertical direction of the frame. This added 



ability allowed the system to distinguish objects that formed varying angles with the surface of travel as well as its 
horizontal orientation to the object 


Horizontal relation to the target was determined by measuring multiple points across the width of the image. Knowing 
the distance to these points and the horizontal separation in the image allows the angle of die camera in relation to the 
target to be determined on a single axis. The technique is extended to include the ability to measure relative angles on 
the vertical axis. 

Additional points are introduced into the projected pattern to provide this extended reference. The calibration routine is 
required to account for this additional information and ultimately the final system will put the new data to use. The cost 
of calculating the additional measurements can be prohibitive depending on the size requirements of the final device and 
the choice of processor chosen for implementation. In the case of die TMS3206416 die additional calculations had 
minimal inpact on the performance of the system and structure of the implementation. As such, the ability to measure 
the relative roll as well as yaw with respect to the target proves a cost effective addition to the abilities of the system. 

4.4 Demonstration of Navigation 

A means of demonstrating the vision system’s capability in a practical setting was developed. A robotic platform was 
controlled using data from the range sensor and made to navigate an environment that was confined in relation to the 
overall size of die device. 



Figure 5. Miniature robotic platform Figured. Scale image of laser used 


in miniature robotic platform 

The platform, illustrated in figures 5 and 6, measured 5 inches wide and 3 inches long. Height of the device was not a 
concern given that the demonstration was two-dimensional. The test environment was created to allow an inch on either 
side of the platform both for error and allowing the ability to turn in the same space. While turning in a corridor die 
limiting size factor become the diagonal length of the robot. The routine the robot was designed to execute 
demonstrates its ability to determine not only distance to an object but the orientation of the system to the target. Using 
this data as feedback it is possible to propel the system through an environment in a controlled fashion with consistency. 
The controllability and consistency of this technique, as demonstrated using the robotic platform, make it ideal for the 
purpose of inspection. 




5. RESULTS 


Currently, the resolution of the camera is the limiting factor in determining die precision of the vision system. In tests 
with small format s ensors the resolution was not significantly reduced with sensors dimensions as small as Vi inch. 
Each implementation thus far has depended on an analog camera and is therefore bound by the resolution of the NTSC 
standard, 720x480 pixels. The use of a higher resolution, low frame-rate, digital camera that provides digital output 
would not only increase the precision of the system but also remove the need for a frame grabber, furthering the 
miniaturization of the overall system. This greater precision would directly result in improved accuracy. 

With mission safety being scrutinized more closely than ever, steps must be taken to improve the processes with which 
we ensure reliable, safe operation. T he design of this vision system addresses deficiencies in the ability to deliver 
diagnostic instrumentation into a confined area with precision and consistency. These abilities will allow for inspection 
with greater frequency and of a less destructive nature. The inclusion of an expanded pattern being projected onto the 
target allows for additional information to be extracted from each frame without a significant increase in processing 
requirements. It is these attributes, along with the ability to miniaturize the system, that make it ideal for the purpose of 
inspection. 
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